chore(main): release 0.38.0 - #604

🤖 I have created a release beep boop

0.38.0 (2026-10-17)

Features

• extract: configurable LLM timeout for JSON extraction (fa65e80)
• renderer: configurable residential proxy username format (808f654)

Bug Fixes

• batch: keep a refused .pdf page in batch, as /v1/scrape does (14ea8a7)
• ci: authenticate the legacy image mirror with a PAT (55c2056)
• config: honour extraction.only_main_content as the request default (53a6baf), closes #614
• core: log target URLs as origin plus a keyed tag (a0ef20b)
• crawl: record a wall no renderer tier could clear as a failed page (f18a3cb)
• crawl: report the real timeout and stop crawling binary assets (df29f00)
• crawl: stop logging a renderer verdict as an anti-bot wall (e173c0c)
• extract: current Anthropic models as defaults, parse thinking blocks (5590aad)
• map: read no links or sitemap entries from a carried wall (7d124b8)
• mcp: keep structural and parked pages as data, like the CLI (f6a80bb)
• mcp: read only a scrape's failure envelope as a tool error (939ba77)
• renderer: a ladder that never reached the wall keeps its error (5a12321)
• renderer: give the PDF gate an honest verdict and keep /v1 as main (4624c9e)
• renderer: launch headless Chrome with --use-mock-keychain (f12e95b)
• renderer: skip a body tag inside head scripts when measuring body text (baacc11), closes #605
• renderer: stop closing Web Worker targets mid-render (3622b86)
• renderer: stop reporting an anti-bot wall as a gateway failure (ba8610c)
• renderer: stop sending failed PDF fetches to browser tiers (4bf631d)
• renderer: stop sending LightPanda a Mozilla-stripped user agent (4f5c6c0)
• renderer: treat a .pdf hard-block status as an unclearable wall (30622db)
• scrape: carry the block verdict correctly on every surface (7f154b2)
• scrape: never accept or re-run an escalation over a condemned wall (8caac65)
• scrape: skip change tracking on a blocked page (bb9c292)
• scrape: skip change tracking only on a wall no tier could clear (60c19e3)
• search: drop enrichment bodies only for a wall no tier could clear (31e6e26)
• server: log every request with its route, status and error code (a498ed7)
• v2: keep a wall no renderer tier could clear unbilled on a 4xx (834a9f1)

This PR was generated with Release Please. See documentation.